Knowledge-Level Planning for Robot Task Planning and Human-Robot Interaction

A robot operating in a real-world domain often needs to do so with incomplete information about the state of the world. A robot with the ability to sense the world can also gather information to generate plans with contingencies, allowing it to reason about the outcome of sensed data at plan time. Moreover, the type of information sensed can vary greatly between domains; for instance, domains involving object ma-nipulation may require sensing information on object location, orientation, and distance, while domains requiring human-robot interaction may involve the sensing of social signals like gaze, facial expression, and language. In this poster, we present some of our recent applications of high-level symbolic planning with incomplete information and sensing actions applied to the problems of task planning and interaction in robotics domains. In particular, buildin

Extending Knowledge-Level Contingent Planning for Robot Task Planning

We present a set of extensions to the knowledge-level PKS (Planning with Knowledge and Sensing) planner, aimed at improving its ability to generate plans in real-world robotics domains. These extensions include a fa-cility for integrating externally-defined reasoning pro-cesses in PKS (e.g., invoking a motion planner), an interval-based fluent representation for capturing the ef-fects of noisy sensors and effectors, and an application programming interface (API) to facilitate software in-tegration on robot platforms. We demonstrate our tech-niques in three simple robot domains, which show their applicability to a broad range of robot planning applica-tions involving incomplete knowledge, real-world ge-ometry, and multiple robots and sensors

Automatically Classifying User Engagement for Dynamic Multi-party Human–Robot Interaction

© 2017, The Author(s). A robot agent designed to engage in real-world human–robot joint action must be able to understand the social states of the human users it interacts with in order to behave appropriately. In particular, in a dynamic public space, a crucial task for the robot is to determine the needs and intentions of all of the people in the scene, so that it only interacts with people who intend to interact with it. We address the task of estimating the engagement state of customers for a robot bartender based on the data from audiovisual sensors. We begin with an offline experiment using hidden Markov models, confirming that the sensor data contains the information necessary to estimate user state. We then present two strategies for online state estimation: a rule-based classifier based on observed human behaviour in real bars, and a set of supervised classifiers trained on a labelled corpus. These strategies are compared in offline cross-validation, in an online user study, and through validation against a separate test corpus. These studies show that while the trained classifiers are best in a cross-validation setting, the rule-based classifier performs best with novel data; however, all classifiers also change their estimate too frequently for practical use. To address this issue, we present a final classifier based on Conditional Random Fields: this model has comparable performance on the test data, with increased stability. In summary, though, the rule-based classifier shows competitive performance with the trained classifiers, suggesting that for this task, such a simple model could actually be a preferred option, providing useful online performance while avoiding the implementation and data-scarcity issues involved in using machine learning for this task

KVP: A knowledge of volumes approach to robot task planning

Abstract — Robot task planning is an inherently challenging problem, as it covers both continuous-space geometric reasoning about robot motion and perception, as well as purely symbolic knowledge about actions and objects. This paper presents a novel “knowledge of volumes ” framework for solving generic robot tasks in partially known environments. In particular, this approach (abbreviated, KVP) combines the power of symbolic, knowledge-level AI planning with the efficient computation of volumes, which serve as an intermediate representation for both robot action and perception. While we demonstrate the effectiveness of our framework in a bimanual robot bartender scenario, our approach is also more generally applicable to tasks in automation and mobile manipulation, involving arbitrary numbers of manipulators. I